Spurious Sentience

On the Unsurprising Finding of Patterns in Latent Spaces

Delft University of Technology

December 6, 2023

Intro

No issue with the methodological ideas that form the foundation of the article in question.
Surprised that people are surprised by the findings:

If we agree that LLMs acquire strong capabilities based on observing information, then where exactly do you expect this information to be stored if not in the parameters of the model?

Spurious Sentience

Spurious relationships: purely associational relationships between two or more variables that are not causally related to each other at all.
Spurious sentience: patterns exhibited by AI that may hint at sentience but are just reflections of the data used to train them.

Example: Principal Component Analysis

Yield Curve

Code

df = CSV.read(joinpath(BLOG_DIR, "data/ust_yields.csv"), DataFrame) |>
    x -> @pivot_longer(x, -Date) |>
    x -> @mutate(x, variable=to_year(variable)) |>
    x -> @mutate(x, year=Dates.year(Date)) |>
    x -> @mutate(x, quarter=Dates.quarter(Date)) |>
    x -> @mutate(x, Date=Dates.format(Date, "yyyy-mm-dd")) |>
    x -> @arrange(x, Date) |>
    x -> @fill_missing(x, "down")
ylims = extrema(skipmissing(df.value))

# Peak-crisis:
onset_date = "2007-02-27"
plt_df = df[df.Date .== onset_date, :]
plt = plot(
    plt_df.variable, plt_df.value;
    label="", color=:blue,
    xlabel="Maturity (years)", ylabel="Yield (%)",
    size=(500,400)
)
scatter!(
    plt_df.variable, plt_df.value;
    label="", color=:blue, alpha=0.5,
    ylims=(0,6)
)
display(plt)

# Post-crisis:
aftermath_date = "2009-04-20"
plt_df = df[df.Date .== aftermath_date, :]
plt = plot(
    plt_df.variable, plt_df.value;
    label="", color=:blue,
    xlabel="Maturity (years)", ylabel="Yield (%)",
    size=(500,400)
)
scatter!(
    plt_df.variable, plt_df.value;
    label="", color=:blue, alpha=0.5,
    ylims=(0,6)
)
display(plt)

(a) Onset of GFC: 27 February 2007.

(b) Aftermath of GFC: 20 April 2009.

Figure 1: Yield curve of US Treasury bonds.

Latent Embeddings vs. Observed Factors (1)

Code

# PCA:
df_wide = @select(df, Date, variable, value) |>
    x -> @pivot_wider(x, names_from = variable, values_from = value) |>
    x -> dropmissing(x)
X = @select(df_wide, -Date) |> Matrix
U, Σ, V = svd(X)
dates = Date.(df_wide.Date)
tick_years = Date.(unique(Dates.year.(dates)))
date_tick = Dates.format.(tick_years, "yyyy")
n_pc = 2
plt_pc = plot(
    dates,
    .-U[:,1:n_pc],
    label=["PC $i" for i in 1:n_pc] |> permutedims,
    size=(1000, 300),
    ylims=(-0.015,0.03),
    legend=:topright
)
plot!(xticks=(tick_years,date_tick), xtickfontsize=6, yaxis=(formatter=y->@sprintf("%.2f",y)))
vline!(Date.([onset_date]), ls=:solid, color=:black, label="Onset of GFC")
vline!(Date.([aftermath_date]), ls=:dash, color=:black, label="Aftermath of GFC")

# Level:
df_level = @group_by(df, Date) |>
    x -> @mutate(x, level=sum(value)/length(value)) |>
    x -> @ungroup(x) |>
    x -> @select(x, Date, level)

# Spreads:
df_spread = @filter(df, variable==0.25 || variable==10) |>
    x -> @select(x, -(year:quarter)) |>
    x -> @mutate(x, variable=ifelse(variable==0.25,"short","long")) |>
    x -> @pivot_wider(x, names_from=variable, values_from=value) |>
    x -> @mutate(x, spread=long-short) |>
    x -> @select(x, Date, spread)

# Plot:
plt_mat = @full_join(df_level, df_spread) |> 
    dropmissing |> 
    unique |>
    x -> @select(x, -Date) |>
    Matrix
plt_obs = plot(
    dates,
    plt_mat,
    label=["Level" "Spread"],
    size=(1000, 300),
    ylims=(-3,9),
    legend=:topright,
    ylab="Yield (%)"
)
plot!(xticks=(tick_years,date_tick), xtickfontsize=6, yaxis=(formatter=y->@sprintf("%.2f",y)))
vline!(Date.([onset_date]), ls=:solid, color=:black, label="Onset of GFC")
vline!(Date.([aftermath_date]), ls=:dash, color=:black, label="Aftermath of GFC")

plot(plt_pc, plt_obs, layout=(2,1), size=(1000, 400), left_margin=5mm, bottom_margin=5mm)

Figure 2: Comparison of latent embeddings and observed data of the US Treasury yield curve.

Latent Embeddings vs. Observed Factors (1)

Figure 3: Yield curve decomposition through PCA.

Example: Deep Learning

Economic Growth and Yield Curve

Code

df_gdp_full = CSV.read(joinpath(BLOG_DIR, "data/gdp.csv"), DataFrame) |>
    x -> @rename(x, Date=DATE, gdp=GDPC1) |>
    x -> @mutate(x, gdp_l1=lag(gdp)) |>
    x -> @mutate(x, growth=log(gdp)-log(gdp_l1)) |>
    x -> @select(x, Date, growth) |>
    x -> @mutate(x, year=Dates.year(Date)) |>
    x -> @mutate(x, quarter=Dates.quarter(Date)) 
df_gdp = df_gdp_full |>
    x -> @filter(x, year <= 2018)

df_yields_qtr = @group_by(df, year, quarter, variable) |>
    x -> @mutate(x, value=mean(value)) |>
    x -> @ungroup(x) |>
    x -> @select(x, -Date) |>
    unique

df_all = @inner_join(df_gdp, df_yields_qtr, (year, quarter)) |> 
    x -> @pivot_wider(x, names_from=variable, values_from=value) |>
    dropmissing

y = df_all.growth |> 
    x -> Float32.(x)
X = @select(df_all, -(Date:quarter)) |> 
    Matrix |>
    x -> Float32.(x) |>
    x -> Flux.normalise(x; dims=1)

# Plot:
p_gdp = plot(
    df_all.Date, y;
    label="", color=:blue,
    size=(800,200),
    ylabel="GDP Growth (log difference)"
)
p_yields = plot(
    df_all.Date, X;
    label="", color=:blue,
    ylabel="Yield (standardized))",
    legend=:bottomright,
    alpha=0.5,
    size=(800,400)
)
plot(p_gdp, p_yields, layout=(2,1), size=(800, 600), left_margin=5mm)

Figure 4: GDP growth and yield curve data.

Autoencoder for Regression

Code

dl = Flux.MLUtils.DataLoader((permutedims(X), permutedims(y)), batchsize=24, shuffle=true)
input_dim = size(X,2)
n_pc = 6
n_hidden = 32
epochs = 1000
activation = tanh_fast
encoder = Flux.Chain(
    Dense(input_dim => n_hidden, activation),
    Dense(n_hidden => n_pc, activation),
) 
decoder = Flux.Chain(
    Dense(n_pc => n_hidden, activation),
    Dense(n_hidden => input_dim, activation),
) 
model = Flux.Chain(
    encoder.layers...,
    decoder.layers...,
    Dense(input_dim, 1),
) 
plt = plot(model, rand(input_dim))
display(plt)

Figure 5: Model architecture.

Code

loss(yhat, y) = Flux.mse(yhat, y)
opt = Adam()
opt_state = Flux.setup(opt, model)
for epoch in 1:epochs
    Flux.train!(model, dl, opt_state) do m, x, y
        loss(m(x), y)
    end
end

Linear Probe - Idea

Linear probe to regress the observed yield curve factors on the latent embeddings.
Let \(F_t\) denote the vector containing the two factors of interest in time \(t\): \(f_t^{\text{spread}}\) and \(f_t^{\text{level}}\).
Formally, we are interested in the following regression model: \(p_{w}(F_t|\mathbf{A}_t)\) where \(w\) denotes the regression parameters.
Following [TEGMARK PAPER REFERENCE] we use Ridge regression with \(\lambda\) set to \(0.1\).
Predict the yield curve factors from the latent embeddings: \(\hat{F}_t=\hat{w}^{\prime}\mathbf{A}_t\).

Linear Probe - Results

Code

yhat = model(X')' 
plt_gdp = plot(
    df_all.Date, y;
    label="Actual", color=:green,
    title="Modelling GDP Growth",
)
plot!(
    df_all.Date, yhat; 
    label="Predicted", color=:green, ls=:dash,
)

λ = 0.1
Y = df_factors[:, [:spread, :level]] |> Matrix
A = encoder(X')'
W = (A'A + UniformScaling(λ)) \ A'Y
Ŷ = A * W
plt_probe = plot(
    df_all.Date, Y;
    color=[:blue :orange],
    label=["Spread" "Level"],
    title="Linear Probe"
)
plot!(
    df_all.Date, Ŷ; 
    color=[:blue :orange], ls=:dash, 
    label=["Spread (predicted)" "Level (predicted)"]
)
plot(plt_gdp, plt_probe, layout=(2,1), size=(800, 600), left_margin=5mm)

Figure 6: The top panel shows the actual GDP growth and fitted values from the autoencoder model. The bottom panel shows the observed average level and spread of the yield curve (solid) along with the predicted values from the linear probe based on the latent embedding (dashed).

What’s the Point?

Demonstrates that the black-box model has learned plausible explanations for the data.
Beyond that, patterns in the latent space might actually be useful for downstream tasks.

Example: gEnEraTiVe Ai

MNIST VAE

Train a tokenizer that can map from the entities to the latent spac \(A\) of the VAE.
Map back from VAE to entities and use another pretrained MLP to predict labels.
Linear probe from \(A\) to geographical location.

This tweet inspired me. Just as I had feared, even the small VAE I trained on MNIST a while ago is basically sentient. It is so over people!https://t.co/gQ0sLvhkkS https://t.co/cANQShuXIK pic.twitter.com/aHycQ6q9pv
— Pat Alt ((paltmey?)) October 5, 2023

Spurious Sentience

Intro

Spurious Sentience

Example: Principal Component Analysis

Yield Curve

Latent Embeddings vs. Observed Factors (1)

Latent Embeddings vs. Observed Factors (1)

Example: Deep Learning

Economic Growth and Yield Curve

Autoencoder for Regression

Linear Probe - Idea

Linear Probe - Results

What’s the Point?

Example: gEnEraTiVe Ai

MNIST VAE

References